Autogenerated HTML docs for v1.5.0-rc3-175-g6506
diff --git a/git-diff-stages.html b/git-diff-stages.html index 6938a2e..0bbd9cf 100644 --- a/git-diff-stages.html +++ b/git-diff-stages.html
@@ -276,6 +276,7 @@ </div> <h2>DESCRIPTION</h2> <div class="sectionbody"> +<p>DEPRECATED and will be removed in 1.5.1.</p> <p>Compares the content and mode of the blobs in two stages in an unmerged index file.</p> </div> @@ -969,7 +970,7 @@ </div> <div id="footer"> <div id="footer-text"> -Last updated 04-Feb-2007 08:31:43 UTC +Last updated 07-Feb-2007 05:52:24 UTC </div> </div> </body>
diff --git a/git-diff-stages.txt b/git-diff-stages.txt index 120d14e..b8f45b8 100644 --- a/git-diff-stages.txt +++ b/git-diff-stages.txt
@@ -12,6 +12,8 @@ DESCRIPTION ----------- +DEPRECATED and will be removed in 1.5.1. + Compares the content and mode of the blobs in two stages in an unmerged index file.
diff --git a/git-fast-import.html b/git-fast-import.html new file mode 100644 index 0000000..9344f6e --- /dev/null +++ b/git-fast-import.html
@@ -0,0 +1,1050 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" + "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> +<meta name="generator" content="AsciiDoc 7.0.2" /> +<style type="text/css"> +/* Debug borders */ +p, li, dt, dd, div, pre, h1, h2, h3, h4, h5, h6 { +/* + border: 1px solid red; +*/ +} + +body { + margin: 1em 5% 1em 5%; +} + +a { color: blue; } +a:visited { color: fuchsia; } + +em { + font-style: italic; +} + +strong { + font-weight: bold; +} + +tt { + color: navy; +} + +h1, h2, h3, h4, h5, h6 { + color: #527bbd; + font-family: sans-serif; + margin-top: 1.2em; + margin-bottom: 0.5em; + line-height: 1.3; +} + +h1 { + border-bottom: 2px solid silver; +} +h2 { + border-bottom: 2px solid silver; + padding-top: 0.5em; +} + +div.sectionbody { + font-family: serif; + margin-left: 0; +} + +hr { + border: 1px solid silver; +} + +p { + margin-top: 0.5em; + margin-bottom: 0.5em; +} + +pre { + padding: 0; + margin: 0; +} + +span#author { + color: #527bbd; + font-family: sans-serif; + font-weight: bold; + font-size: 1.2em; +} +span#email { +} +span#revision { + font-family: sans-serif; +} + +div#footer { + font-family: sans-serif; + font-size: small; + border-top: 2px solid silver; + padding-top: 0.5em; + margin-top: 4.0em; +} +div#footer-text { + float: left; + padding-bottom: 0.5em; +} +div#footer-badges { + float: right; + padding-bottom: 0.5em; +} + +div#preamble, +div.tableblock, div.imageblock, div.exampleblock, div.verseblock, +div.quoteblock, div.literalblock, div.listingblock, div.sidebarblock, +div.admonitionblock { + margin-right: 10%; + margin-top: 1.5em; + margin-bottom: 1.5em; +} +div.admonitionblock { + margin-top: 2.5em; + margin-bottom: 2.5em; +} + +div.content { /* Block element content. */ + padding: 0; +} + +/* Block element titles. */ +div.title, caption.title { + font-family: sans-serif; + font-weight: bold; + text-align: left; + margin-top: 1.0em; + margin-bottom: 0.5em; +} +div.title + * { + margin-top: 0; +} + +td div.title:first-child { + margin-top: 0.0em; +} +div.content div.title:first-child { + margin-top: 0.0em; +} +div.content + div.title { + margin-top: 0.0em; +} + +div.sidebarblock > div.content { + background: #ffffee; + border: 1px solid silver; + padding: 0.5em; +} + +div.listingblock > div.content { + border: 1px solid silver; + background: #f4f4f4; + padding: 0.5em; +} + +div.quoteblock > div.content { + padding-left: 2.0em; +} +div.quoteblock .attribution { + text-align: right; +} + +div.admonitionblock .icon { + vertical-align: top; + font-size: 1.1em; + font-weight: bold; + text-decoration: underline; + color: #527bbd; + padding-right: 0.5em; +} +div.admonitionblock td.content { + padding-left: 0.5em; + border-left: 2px solid silver; +} + +div.exampleblock > div.content { + border-left: 2px solid silver; + padding: 0.5em; +} + +div.verseblock div.content { + white-space: pre; +} + +div.imageblock div.content { padding-left: 0; } +div.imageblock img { border: 1px solid silver; } +span.image img { border-style: none; } + +dl { + margin-top: 0.8em; + margin-bottom: 0.8em; +} +dt { + margin-top: 0.5em; + margin-bottom: 0; + font-style: italic; +} +dd > *:first-child { + margin-top: 0; +} + +ul, ol { + list-style-position: outside; +} +ol.olist2 { + list-style-type: lower-alpha; +} + +div.tableblock > table { + border-color: #527bbd; + border-width: 3px; +} +thead { + font-family: sans-serif; + font-weight: bold; +} +tfoot { + font-weight: bold; +} + +div.hlist { + margin-top: 0.8em; + margin-bottom: 0.8em; +} +td.hlist1 { + vertical-align: top; + font-style: italic; + padding-right: 0.8em; +} +td.hlist2 { + vertical-align: top; +} + +@media print { + div#footer-badges { display: none; } +} +include::./stylesheets/xhtml11-manpage.css[] +/* Workarounds for IE6's broken and incomplete CSS2. */ + +div.sidebar-content { + background: #ffffee; + border: 1px solid silver; + padding: 0.5em; +} +div.sidebar-title, div.image-title { + font-family: sans-serif; + font-weight: bold; + margin-top: 0.0em; + margin-bottom: 0.5em; +} + +div.listingblock div.content { + border: 1px solid silver; + background: #f4f4f4; + padding: 0.5em; +} + +div.quoteblock-content { + padding-left: 2.0em; +} + +div.exampleblock-content { + border-left: 2px solid silver; + padding-left: 0.5em; +} +</style> +<title>git-fast-import(1)</title> +</head> +<body> +<div id="header"> +<h1> +git-fast-import(1) Manual Page +</h1> +<h2>NAME</h2> +<div class="sectionbody"> +<p>git-fast-import - + Backend for fast Git data importers. +</p> +</div> +</div> +<h2>SYNOPSIS</h2> +<div class="sectionbody"> +<p>frontend | <em>git-fast-import</em> [options]</p> +</div> +<h2>DESCRIPTION</h2> +<div class="sectionbody"> +<p>This program is usually not what the end user wants to run directly. +Most end users want to use one of the existing frontend programs, +which parses a specific type of foreign source and feeds the contents +stored there to git-fast-import (gfi).</p> +<p>gfi reads a mixed command/data stream from standard input and +writes one or more packfiles directly into the current repository. +When EOF is received on standard input, fast import writes out +updated branch and tag refs, fully updating the current repository +with the newly imported data.</p> +<p>The gfi backend itself can import into an empty repository (one that +has already been initialized by <a href="git-init.html">git-init(1)</a>) or incrementally +update an existing populated repository. Whether or not incremental +imports are supported from a particular foreign source depends on +the frontend program in use.</p> +</div> +<h2>OPTIONS</h2> +<div class="sectionbody"> +<dl> +<dt> +--date-format=<fmt> +</dt> +<dd> +<p> + Specify the type of dates the frontend will supply to + gfi within <tt>author</tt>, <tt>committer</tt> and <tt>tagger</tt> commands. + See “Date Formats” below for details about which formats + are supported, and their syntax. +</p> +</dd> +<dt> +--force +</dt> +<dd> +<p> + Force updating modified existing branches, even if doing + so would cause commits to be lost (as the new commit does + not contain the old commit). +</p> +</dd> +<dt> +--max-pack-size=<n> +</dt> +<dd> +<p> + Maximum size of each output packfile, expressed in MiB. + The default is 4096 (4 GiB) as that is the maximum allowed + packfile size (due to file format limitations). Some + importers may wish to lower this, such as to ensure the + resulting packfiles fit on CDs. +</p> +</dd> +<dt> +--depth=<n> +</dt> +<dd> +<p> + Maximum delta depth, for blob and tree deltification. + Default is 10. +</p> +</dd> +<dt> +--active-branches=<n> +</dt> +<dd> +<p> + Maximum number of branches to maintain active at once. + See “Memory Utilization” below for details. Default is 5. +</p> +</dd> +<dt> +--export-marks=<file> +</dt> +<dd> +<p> + Dumps the internal marks table to <file> when complete. + Marks are written one per line as <tt>:markid SHA-1</tt>. + Frontends can use this file to validate imports after they + have been completed. +</p> +</dd> +</dl> +</div> +<h2>Performance</h2> +<div class="sectionbody"> +<p>The design of gfi allows it to import large projects in a minimum +amount of memory usage and processing time. Assuming the frontend +is able to keep up with gfi and feed it a constant stream of data, +import times for projects holding 10+ years of history and containing +100,000+ individual commits are generally completed in just 1-2 +hours on quite modest (~$2,000 USD) hardware.</p> +<p>Most bottlenecks appear to be in foreign source data access (the +source just cannot extract revisions fast enough) or disk IO (gfi +writes as fast as the disk will take the data). Imports will run +faster if the source data is stored on a different drive than the +destination Git repository (due to less IO contention).</p> +</div> +<h2>Development Cost</h2> +<div class="sectionbody"> +<p>A typical frontend for gfi tends to weigh in at approximately 200 +lines of Perl/Python/Ruby code. Most developers have been able to +create working importers in just a couple of hours, even though it +is their first exposure to gfi, and sometimes even to Git. This is +an ideal situation, given that most conversion tools are throw-away +(use once, and never look back).</p> +</div> +<h2>Parallel Operation</h2> +<div class="sectionbody"> +<p>Like <tt>git-push</tt> or <tt>git-fetch</tt>, imports handled by gfi are safe to +run alongside parallel <tt>git repack -a -d</tt> or <tt>git gc</tt> invocations, +or any other Git operation (including <tt>git prune</tt>, as loose objects +are never used by gfi).</p> +<p>gfi does not lock the branch or tag refs it is actively importing. +After the import, during its ref update phase, gfi tests each +existing branch ref to verify the update will be a fast-forward +update (the commit stored in the ref is contained in the new +history of the commit to be written). If the update is not a +fast-forward update, gfi will skip updating that ref and instead +prints a warning message. gfi will always attempt to update all +branch refs, and does not stop on the first failure.</p> +<p>Branch updates can be forced with <tt>--force</tt>, but its recommended that +this only be used on an otherwise quiet repository. Using <tt>--force</tt> +is not necessary for an initial import into an empty repository.</p> +</div> +<h2>Technical Discussion</h2> +<div class="sectionbody"> +<p>gfi tracks a set of branches in memory. Any branch can be created +or modified at any point during the import process by sending a +<tt>commit</tt> command on the input stream. This design allows a frontend +program to process an unlimited number of branches simultaneously, +generating commits in the order they are available from the source +data. It also simplifies the frontend programs considerably.</p> +<p>gfi does not use or alter the current working directory, or any +file within it. (It does however update the current Git repository, +as referenced by <tt>GIT_DIR</tt>.) Therefore an import frontend may use +the working directory for its own purposes, such as extracting file +revisions from the foreign source. This ignorance of the working +directory also allows gfi to run very quickly, as it does not +need to perform any costly file update operations when switching +between branches.</p> +</div> +<h2>Input Format</h2> +<div class="sectionbody"> +<p>With the exception of raw file data (which Git does not interpret) +the gfi input format is text (ASCII) based. This text based +format simplifies development and debugging of frontend programs, +especially when a higher level language such as Perl, Python or +Ruby is being used.</p> +<p>gfi is very strict about its input. Where we say SP below we mean +<strong>exactly</strong> one space. Likewise LF means one (and only one) linefeed. +Supplying additional whitespace characters will cause unexpected +results, such as branch names or file names with leading or trailing +spaces in their name, or early termination of gfi when it encounters +unexpected input.</p> +<h3>Date Formats</h3> +<p>The following date formats are supported. A frontend should select +the format it will use for this import by passing the format name +in the <tt>--date-format=<fmt></tt> command line option.</p> +<dl> +<dt> +<tt>raw</tt> +</dt> +<dd> +<p> + This is the Git native format and is <tt><time> SP <tz></tt>. + It is also gfi's default format, if <tt>--date-format</tt> was + not specified. +</p> +<p>The time of the event is specified by <tt><time></tt> as the number of +seconds since the UNIX epoch (midnight, Jan 1, 1970, UTC) and is +written as an ASCII decimal integer.</p> +<p>The timezone is specified by <tt><tz></tt> as a positive or negative offset +from UTC. For example EST (which is typically 5 hours behind GMT) +would be expressed in <tt><tz></tt> by “-0500” while GMT is “+0000”.</p> +<p>If the timezone is not available in the source material, use +“+0000”, or the most common local timezone. For example many +organizations have a CVS repository which has only ever been accessed +by users who are located in the same location and timezone. In this +case the user's timezone can be easily assumed.</p> +<p>Unlike the <tt>rfc2822</tt> format, this format is very strict. Any +variation in formatting will cause gfi to reject the value.</p> +</dd> +<dt> +<tt>rfc2822</tt> +</dt> +<dd> +<p> + This is the standard email format as described by RFC 2822. +</p> +<p>An example value is “Tue Feb 6 11:22:18 2007 -0500”. The Git +parser is accurate, but a little on the lenient side. Its the +same parser used by <a href="git-am.html">git-am(1)</a> when applying patches +received from email.</p> +<p>Some malformed strings may be accepted as valid dates. In some of +these cases Git will still be able to obtain the correct date from +the malformed string. There are also some types of malformed +strings which Git will parse wrong, and yet consider valid. +Seriously malformed strings will be rejected.</p> +<p>If the source material is formatted in RFC 2822 style dates, +the frontend should let gfi handle the parsing and conversion +(rather than attempting to do it itself) as the Git parser has +been well tested in the wild.</p> +<p>Frontends should prefer the <tt>raw</tt> format if the source material +is already in UNIX-epoch format, or is easily convertible to +that format, as there is no ambiguity in parsing.</p> +</dd> +<dt> +<tt>now</tt> +</dt> +<dd> +<p> + Always use the current time and timezone. The literal + <tt>now</tt> must always be supplied for <tt><when></tt>. +</p> +<p>This is a toy format. The current time and timezone of this system +is always copied into the identity string at the time it is being +created by gfi. There is no way to specify a different time or +timezone.</p> +<p>This particular format is supplied as its short to implement and +may be useful to a process that wants to create a new commit +right now, without needing to use a working directory or +<a href="git-update-index.html">git-update-index(1)</a>.</p> +<p>If separate <tt>author</tt> and <tt>committer</tt> commands are used in a <tt>commit</tt> +the timestamps may not match, as the system clock will be polled +twice (once for each command). The only way to ensure that both +author and committer identity information has the same timestamp +is to omit <tt>author</tt> (thus copying from <tt>committer</tt>) or to use a +date format other than <tt>now</tt>.</p> +</dd> +</dl> +<h3>Commands</h3> +<p>gfi accepts several commands to update the current repository +and control the current import process. More detailed discussion +(with examples) of each command follows later.</p> +<dl> +<dt> +<tt>commit</tt> +</dt> +<dd> +<p> + Creates a new branch or updates an existing branch by + creating a new commit and updating the branch to point at + the newly created commit. +</p> +</dd> +<dt> +<tt>tag</tt> +</dt> +<dd> +<p> + Creates an annotated tag object from an existing commit or + branch. Lightweight tags are not supported by this command, + as they are not recommended for recording meaningful points + in time. +</p> +</dd> +<dt> +<tt>reset</tt> +</dt> +<dd> +<p> + Reset an existing branch (or a new branch) to a specific + revision. This command must be used to change a branch to + a specific revision without making a commit on it. +</p> +</dd> +<dt> +<tt>blob</tt> +</dt> +<dd> +<p> + Convert raw file data into a blob, for future use in a + <tt>commit</tt> command. This command is optional and is not + needed to perform an import. +</p> +</dd> +<dt> +<tt>checkpoint</tt> +</dt> +<dd> +<p> + Forces gfi to close the current packfile, generate its + unique SHA-1 checksum and index, and start a new packfile. + This command is optional and is not needed to perform + an import. +</p> +</dd> +</dl> +<h3><tt>commit</tt></h3> +<p>Create or update a branch with a new commit, recording one logical +change to the project.</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'commit' SP <ref> LF + mark? + ('author' SP <name> SP LT <email> GT SP <when> LF)? + 'committer' SP <name> SP LT <email> GT SP <when> LF + data + ('from' SP <committish> LF)? + ('merge' SP <committish> LF)? + (filemodify | filedelete)* + LF</tt></pre> +</div></div> +<p>where <tt><ref></tt> is the name of the branch to make the commit on. +Typically branch names are prefixed with <tt>refs/heads/</tt> in +Git, so importing the CVS branch symbol <tt>RELENG-1_0</tt> would use +<tt>refs/heads/RELENG-1_0</tt> for the value of <tt><ref></tt>. The value of +<tt><ref></tt> must be a valid refname in Git. As <tt>LF</tt> is not valid in +a Git refname, no quoting or escaping syntax is supported here.</p> +<p>A <tt>mark</tt> command may optionally appear, requesting gfi to save a +reference to the newly created commit for future use by the frontend +(see below for format). It is very common for frontends to mark +every commit they create, thereby allowing future branch creation +from any imported commit.</p> +<p>The <tt>data</tt> command following <tt>committer</tt> must supply the commit +message (see below for <tt>data</tt> command syntax). To import an empty +commit message use a 0 length data. Commit messages are free-form +and are not interpreted by Git. Currently they must be encoded in +UTF-8, as gfi does not permit other encodings to be specified.</p> +<p>Zero or more <tt>filemodify</tt> and <tt>filedelete</tt> commands may be +included to update the contents of the branch prior to the commit. +These commands can be supplied in any order, gfi is not sensitive +to pathname or operation ordering.</p> +<h4><tt>author</tt></h4> +<p>An <tt>author</tt> command may optionally appear, if the author information +might differ from the committer information. If <tt>author</tt> is omitted +then gfi will automatically use the committer's information for +the author portion of the commit. See below for a description of +the fields in <tt>author</tt>, as they are identical to <tt>committer</tt>.</p> +<h4><tt>committer</tt></h4> +<p>The <tt>committer</tt> command indicates who made this commit, and when +they made it.</p> +<p>Here <tt><name></tt> is the person's display name (for example +“Com M Itter”) and <tt><email></tt> is the person's email address +(“cm@example.com”). <tt>LT</tt> and <tt>GT</tt> are the literal less-than (\x3c) +and greater-than (\x3e) symbols. These are required to delimit +the email address from the other fields in the line. Note that +<tt><name></tt> is free-form and may contain any sequence of bytes, except +<tt>LT</tt> and <tt>LF</tt>. It is typically UTF-8 encoded.</p> +<p>The time of the change is specified by <tt><when></tt> using the date format +that was selected by the <tt>--date-format=<fmt></tt> command line option. +See “Date Formats” above for the set of supported formats, and +their syntax.</p> +<h4><tt>from</tt></h4> +<p>Only valid for the first commit made on this branch by this +gfi process. The <tt>from</tt> command is used to specify the commit +to initialize this branch from. This revision will be the first +ancestor of the new commit.</p> +<p>Omitting the <tt>from</tt> command in the first commit of a new branch will +cause gfi to create that commit with no ancestor. This tends to be +desired only for the initial commit of a project. Omitting the +<tt>from</tt> command on existing branches is required, as the current +commit on that branch is automatically assumed to be the first +ancestor of the new commit.</p> +<p>As <tt>LF</tt> is not valid in a Git refname or SHA-1 expression, no +quoting or escaping syntax is supported within <tt><committish></tt>.</p> +<p>Here <tt><committish></tt> is any of the following:</p> +<ul> +<li> +<p> +The name of an existing branch already in gfi's internal branch + table. If gfi doesn't know the name, its treated as a SHA-1 + expression. +</p> +</li> +<li> +<p> +A mark reference, <tt>:<idnum></tt>, where <tt><idnum></tt> is the mark number. +</p> +<p>The reason gfi uses <tt>:</tt> to denote a mark reference is this character +is not legal in a Git branch name. The leading <tt>:</tt> makes it easy +to distingush between the mark 42 (<tt>:42</tt>) and the branch 42 (<tt>42</tt> +or <tt>refs/heads/42</tt>), or an abbreviated SHA-1 which happened to +consist only of base-10 digits.</p> +<p>Marks must be declared (via <tt>mark</tt>) before they can be used.</p> +</li> +<li> +<p> +A complete 40 byte or abbreviated commit SHA-1 in hex. +</p> +</li> +<li> +<p> +Any valid Git SHA-1 expression that resolves to a commit. See + “SPECIFYING REVISIONS” in <a href="git-rev-parse.html">git-rev-parse(1)</a> for details. +</p> +</li> +</ul> +<p>The special case of restarting an incremental import from the +current branch value should be written as:</p> +<div class="listingblock"> +<div class="content"> +<pre><tt> from refs/heads/branch^0</tt></pre> +</div></div> +<p>The <tt>^0</tt> suffix is necessary as gfi does not permit a branch to +start from itself, and the branch is created in memory before the +<tt>from</tt> command is even read from the input. Adding <tt>^0</tt> will force +gfi to resolve the commit through Git's revision parsing library, +rather than its internal branch table, thereby loading in the +existing value of the branch.</p> +<h4><tt>merge</tt></h4> +<p>Includes one additional ancestor commit, and makes the current +commit a merge commit. An unlimited number of <tt>merge</tt> commands per +commit are permitted by gfi, thereby establishing an n-way merge. +However Git's other tools never create commits with more than 15 +additional ancestors (forming a 16-way merge). For this reason +it is suggested that frontends do not use more than 15 <tt>merge</tt> +commands per commit.</p> +<p>Here <tt><committish></tt> is any of the commit specification expressions +also accepted by <tt>from</tt> (see above).</p> +<h4><tt>filemodify</tt></h4> +<p>Included in a <tt>commit</tt> command to add a new file or change the +content of an existing file. This command has two different means +of specifying the content of the file.</p> +<dl> +<dt> +External data format +</dt> +<dd> +<p> + The data content for the file was already supplied by a prior + <tt>blob</tt> command. The frontend just needs to connect it. +</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'M' SP <mode> SP <dataref> SP <path> LF</tt></pre> +</div></div> +<p>Here <tt><dataref></tt> can be either a mark reference (<tt>:<idnum></tt>) +set by a prior <tt>blob</tt> command, or a full 40-byte SHA-1 of an +existing Git blob object.</p> +</dd> +<dt> +Inline data format +</dt> +<dd> +<p> + The data content for the file has not been supplied yet. + The frontend wants to supply it as part of this modify + command. +</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'M' SP <mode> SP 'inline' SP <path> LF + data</tt></pre> +</div></div> +<p>See below for a detailed description of the <tt>data</tt> command.</p> +</dd> +</dl> +<p>In both formats <tt><mode></tt> is the type of file entry, specified +in octal. Git only supports the following modes:</p> +<ul> +<li> +<p> +<tt>100644</tt> or <tt>644</tt>: A normal (not-executable) file. The majority + of files in most projects use this mode. If in doubt, this is + what you want. +</p> +</li> +<li> +<p> +<tt>100755</tt> or <tt>755</tt>: A normal, but executable, file. +</p> +</li> +<li> +<p> +<tt>120000</tt>: A symlink, the content of the file will be the link target. +</p> +</li> +</ul> +<p>In both formats <tt><path></tt> is the complete path of the file to be added +(if not already existing) or modified (if already existing).</p> +<p>A <tt><path></tt> string must use UNIX-style directory seperators (forward +slash <tt>/</tt>), may contain any byte other than <tt>LF</tt>, and must not +start with double quote (<tt>"</tt>).</p> +<p>If an <tt>LF</tt> or double quote must be encoded into <tt><path></tt> shell-style +quoting should be used, e.g. <tt>"path/with\n and \" in it"</tt>.</p> +<p>The value of <tt><path></tt> must be in canoncial form. That is it must not:</p> +<ul> +<li> +<p> +contain an empty directory component (e.g. <tt>foo//bar</tt> is invalid), +</p> +</li> +<li> +<p> +end with a directory seperator (e.g. <tt>foo/</tt> is invalid), +</p> +</li> +<li> +<p> +start with a directory seperator (e.g. <tt>/foo</tt> is invalid), +</p> +</li> +<li> +<p> +contain the special component <tt>.</tt> or <tt>..</tt> (e.g. <tt>foo/./bar</tt> and + <tt>foo/../bar</tt> are invalid). +</p> +</li> +</ul> +<p>It is recommended that <tt><path></tt> always be encoded using UTF-8.</p> +<h4><tt>filedelete</tt></h4> +<p>Included in a <tt>commit</tt> command to remove a file from the branch. +If the file removal makes its directory empty, the directory will +be automatically removed too. This cascades up the tree until the +first non-empty directory or the root is reached.</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'D' SP <path> LF</tt></pre> +</div></div> +<p>here <tt><path></tt> is the complete path of the file to be removed. +See <tt>filemodify</tt> above for a detailed description of <tt><path></tt>.</p> +<h3><tt>mark</tt></h3> +<p>Arranges for gfi to save a reference to the current object, allowing +the frontend to recall this object at a future point in time, without +knowing its SHA-1. Here the current object is the object creation +command the <tt>mark</tt> command appears within. This can be <tt>commit</tt>, +<tt>tag</tt>, and <tt>blob</tt>, but <tt>commit</tt> is the most common usage.</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'mark' SP ':' <idnum> LF</tt></pre> +</div></div> +<p>where <tt><idnum></tt> is the number assigned by the frontend to this mark. +The value of <tt><idnum></tt> is expressed as an ASCII decimal integer. +The value 0 is reserved and cannot be used as +a mark. Only values greater than or equal to 1 may be used as marks.</p> +<p>New marks are created automatically. Existing marks can be moved +to another object simply by reusing the same <tt><idnum></tt> in another +<tt>mark</tt> command.</p> +<h3><tt>tag</tt></h3> +<p>Creates an annotated tag referring to a specific commit. To create +lightweight (non-annotated) tags see the <tt>reset</tt> command below.</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'tag' SP <name> LF + 'from' SP <committish> LF + 'tagger' SP <name> SP LT <email> GT SP <when> LF + data + LF</tt></pre> +</div></div> +<p>where <tt><name></tt> is the name of the tag to create.</p> +<p>Tag names are automatically prefixed with <tt>refs/tags/</tt> when stored +in Git, so importing the CVS branch symbol <tt>RELENG-1_0-FINAL</tt> would +use just <tt>RELENG-1_0-FINAL</tt> for <tt><name></tt>, and gfi will write the +corresponding ref as <tt>refs/tags/RELENG-1_0-FINAL</tt>.</p> +<p>The value of <tt><name></tt> must be a valid refname in Git and therefore +may contain forward slashes. As <tt>LF</tt> is not valid in a Git refname, +no quoting or escaping syntax is supported here.</p> +<p>The <tt>from</tt> command is the same as in the <tt>commit</tt> command; see +above for details.</p> +<p>The <tt>tagger</tt> command uses the same format as <tt>committer</tt> within +<tt>commit</tt>; again see above for details.</p> +<p>The <tt>data</tt> command following <tt>tagger</tt> must supply the annotated tag +message (see below for <tt>data</tt> command syntax). To import an empty +tag message use a 0 length data. Tag messages are free-form and are +not interpreted by Git. Currently they must be encoded in UTF-8, +as gfi does not permit other encodings to be specified.</p> +<p>Signing annotated tags during import from within gfi is not +supported. Trying to include your own PGP/GPG signature is not +recommended, as the frontend does not (easily) have access to the +complete set of bytes which normally goes into such a signature. +If signing is required, create lightweight tags from within gfi with +<tt>reset</tt>, then create the annotated versions of those tags offline +with the standard <a href="git-tag.html">git-tag(1)</a> process.</p> +<h3><tt>reset</tt></h3> +<p>Creates (or recreates) the named branch, optionally starting from +a specific revision. The reset command allows a frontend to issue +a new <tt>from</tt> command for an existing branch, or to create a new +branch from an existing commit without creating a new commit.</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'reset' SP <ref> LF + ('from' SP <committish> LF)? + LF</tt></pre> +</div></div> +<p>For a detailed description of <tt><ref></tt> and <tt><committish></tt> see above +under <tt>commit</tt> and <tt>from</tt>.</p> +<p>The <tt>reset</tt> command can also be used to create lightweight +(non-annotated) tags. For example:</p> +<div class="exampleblock"> +<div class="exampleblock-content"> +<div class="literalblock"> +<div class="content"> +<pre><tt>reset refs/tags/938 +from :938</tt></pre> +</div></div> +</div></div> +<p>would create the lightweight tag <tt>refs/tags/938</tt> referring to +whatever commit mark <tt>:938</tt> references.</p> +<h3><tt>blob</tt></h3> +<p>Requests writing one file revision to the packfile. The revision +is not connected to any commit; this connection must be formed in +a subsequent <tt>commit</tt> command by referencing the blob through an +assigned mark.</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'blob' LF + mark? + data</tt></pre> +</div></div> +<p>The mark command is optional here as some frontends have chosen +to generate the Git SHA-1 for the blob on their own, and feed that +directly to <tt>commit</tt>. This is typically more work than its worth +however, as marks are inexpensive to store and easy to use.</p> +<h3><tt>data</tt></h3> +<p>Supplies raw data (for use as blob/file content, commit messages, or +annotated tag messages) to gfi. Data can be supplied using an exact +byte count or delimited with a terminating line. Real frontends +intended for production-quality conversions should always use the +exact byte count format, as it is more robust and performs better. +The delimited format is intended primarily for testing gfi.</p> +<dl> +<dt> +Exact byte count format +</dt> +<dd> +<p> + The frontend must specify the number of bytes of data. +</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'data' SP <count> LF + <raw> LF</tt></pre> +</div></div> +<p>where <tt><count></tt> is the exact number of bytes appearing within +<tt><raw></tt>. The value of <tt><count></tt> is expressed as an ASCII decimal +integer. The <tt>LF</tt> on either side of <tt><raw></tt> is not +included in <tt><count></tt> and will not be included in the imported data.</p> +</dd> +<dt> +Delimited format +</dt> +<dd> +<p> + A delimiter string is used to mark the end of the data. + gfi will compute the length by searching for the delimiter. + This format is primarly useful for testing and is not + recommended for real data. +</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'data' SP '<<' <delim> LF + <raw> LF + <delim> LF</tt></pre> +</div></div> +<p>where <tt><delim></tt> is the chosen delimiter string. The string <tt><delim></tt> +must not appear on a line by itself within <tt><raw></tt>, as otherwise +gfi will think the data ends earlier than it really does. The <tt>LF</tt> +immediately trailing <tt><raw></tt> is part of <tt><raw></tt>. This is one of +the limitations of the delimited format, it is impossible to supply +a data chunk which does not have an LF as its last byte.</p> +</dd> +</dl> +<h3><tt>checkpoint</tt></h3> +<p>Forces gfi to close the current packfile and start a new one. +As this requires a significant amount of CPU time and disk IO +(to compute the overall pack SHA-1 checksum and generate the +corresponding index file) it can easily take several minutes for +a single <tt>checkpoint</tt> command to complete.</p> +<div class="literalblock"> +<div class="content"> +<pre><tt> 'checkpoint' LF + LF</tt></pre> +</div></div> +</div> +<h2>Packfile Optimization</h2> +<div class="sectionbody"> +<p>When packing a blob gfi always attempts to deltify against the last +blob written. Unless specifically arranged for by the frontend, +this will probably not be a prior version of the same file, so the +generated delta will not be the smallest possible. The resulting +packfile will be compressed, but will not be optimal.</p> +<p>Frontends which have efficient access to all revisions of a +single file (for example reading an RCS/CVS ,v file) can choose +to supply all revisions of that file as a sequence of consecutive +<tt>blob</tt> commands. This allows gfi to deltify the different file +revisions against each other, saving space in the final packfile. +Marks can be used to later identify individual file revisions during +a sequence of <tt>commit</tt> commands.</p> +<p>The packfile(s) created by gfi do not encourage good disk access +patterns. This is caused by gfi writing the data in the order +it is received on standard input, while Git typically organizes +data within packfiles to make the most recent (current tip) data +appear before historical data. Git also clusters commits together, +speeding up revision traversal through better cache locality.</p> +<p>For this reason it is strongly recommended that users repack the +repository with <tt>git repack -a -d</tt> after gfi completes, allowing +Git to reorganize the packfiles for faster data access. If blob +deltas are suboptimal (see above) then also adding the <tt>-f</tt> option +to force recomputation of all deltas can significantly reduce the +final packfile size (30-50% smaller can be quite typical).</p> +</div> +<h2>Memory Utilization</h2> +<div class="sectionbody"> +<p>There are a number of factors which affect how much memory gfi +requires to perform an import. Like critical sections of core +Git, gfi uses its own memory allocators to ammortize any overheads +associated with malloc. In practice gfi tends to ammoritize any +malloc overheads to 0, due to its use of large block allocations.</p> +<h3>per object</h3> +<p>gfi maintains an in-memory structure for every object written in +this execution. On a 32 bit system the structure is 32 bytes, +on a 64 bit system the structure is 40 bytes (due to the larger +pointer sizes). Objects in the table are not deallocated until +gfi terminates. Importing 2 million objects on a 32 bit system +will require approximately 64 MiB of memory.</p> +<p>The object table is actually a hashtable keyed on the object name +(the unique SHA-1). This storage configuration allows gfi to reuse +an existing or already written object and avoid writing duplicates +to the output packfile. Duplicate blobs are surprisingly common +in an import, typically due to branch merges in the source.</p> +<h3>per mark</h3> +<p>Marks are stored in a sparse array, using 1 pointer (4 bytes or 8 +bytes, depending on pointer size) per mark. Although the array +is sparse, frontends are still strongly encouraged to use marks +between 1 and n, where n is the total number of marks required for +this import.</p> +<h3>per branch</h3> +<p>Branches are classified as active and inactive. The memory usage +of the two classes is significantly different.</p> +<p>Inactive branches are stored in a structure which uses 96 or 120 +bytes (32 bit or 64 bit systems, respectively), plus the length of +the branch name (typically under 200 bytes), per branch. gfi will +easily handle as many as 10,000 inactive branches in under 2 MiB +of memory.</p> +<p>Active branches have the same overhead as inactive branches, but +also contain copies of every tree that has been recently modified on +that branch. If subtree <tt>include</tt> has not been modified since the +branch became active, its contents will not be loaded into memory, +but if subtree <tt>src</tt> has been modified by a commit since the branch +became active, then its contents will be loaded in memory.</p> +<p>As active branches store metadata about the files contained on that +branch, their in-memory storage size can grow to a considerable size +(see below).</p> +<p>gfi automatically moves active branches to inactive status based on +a simple least-recently-used algorithm. The LRU chain is updated on +each <tt>commit</tt> command. The maximum number of active branches can be +increased or decreased on the command line with <tt>--active-branches=</tt>.</p> +<h3>per active tree</h3> +<p>Trees (aka directories) use just 12 bytes of memory on top of the +memory required for their entries (see “per active file” below). +The cost of a tree is virtually 0, as its overhead ammortizes out +over the individual file entries.</p> +<h3>per active file entry</h3> +<p>Files (and pointers to subtrees) within active trees require 52 or 64 +bytes (32/64 bit platforms) per entry. To conserve space, file and +tree names are pooled in a common string table, allowing the filename +“Makefile” to use just 16 bytes (after including the string header +overhead) no matter how many times it occurs within the project.</p> +<p>The active branch LRU, when coupled with the filename string pool +and lazy loading of subtrees, allows gfi to efficiently import +projects with 2,000+ branches and 45,114+ files in a very limited +memory footprint (less than 2.7 MiB per active branch).</p> +</div> +<h2>Author</h2> +<div class="sectionbody"> +<p>Written by Shawn O. Pearce <spearce@spearce.org>.</p> +</div> +<h2>Documentation</h2> +<div class="sectionbody"> +<p>Documentation by Shawn O. Pearce <spearce@spearce.org>.</p> +</div> +<h2>GIT</h2> +<div class="sectionbody"> +<p>Part of the <a href="git.html">git(7)</a> suite</p> +</div> +<div id="footer"> +<div id="footer-text"> +Last updated 07-Feb-2007 05:52:25 UTC +</div> +</div> +</body> +</html> diff --git a/git-fast-import.txt b/git-fast-import.txt new file mode 100644 index 0000000..1fe2c1d --- /dev/null +++ b/git-fast-import.txt
@@ -0,0 +1,739 @@ +git-fast-import(1) +================== + +NAME +---- +git-fast-import - Backend for fast Git data importers. + + +SYNOPSIS +-------- +frontend | 'git-fast-import' [options] + +DESCRIPTION +----------- +This program is usually not what the end user wants to run directly. +Most end users want to use one of the existing frontend programs, +which parses a specific type of foreign source and feeds the contents +stored there to git-fast-import (gfi). + +gfi reads a mixed command/data stream from standard input and +writes one or more packfiles directly into the current repository. +When EOF is received on standard input, fast import writes out +updated branch and tag refs, fully updating the current repository +with the newly imported data. + +The gfi backend itself can import into an empty repository (one that +has already been initialized by gitlink:git-init[1]) or incrementally +update an existing populated repository. Whether or not incremental +imports are supported from a particular foreign source depends on +the frontend program in use. + + +OPTIONS +------- +--date-format=<fmt>:: + Specify the type of dates the frontend will supply to + gfi within `author`, `committer` and `tagger` commands. + See ``Date Formats'' below for details about which formats + are supported, and their syntax. + +--force:: + Force updating modified existing branches, even if doing + so would cause commits to be lost (as the new commit does + not contain the old commit). + +--max-pack-size=<n>:: + Maximum size of each output packfile, expressed in MiB. + The default is 4096 (4 GiB) as that is the maximum allowed + packfile size (due to file format limitations). Some + importers may wish to lower this, such as to ensure the + resulting packfiles fit on CDs. + +--depth=<n>:: + Maximum delta depth, for blob and tree deltification. + Default is 10. + +--active-branches=<n>:: + Maximum number of branches to maintain active at once. + See ``Memory Utilization'' below for details. Default is 5. + +--export-marks=<file>:: + Dumps the internal marks table to <file> when complete. + Marks are written one per line as `:markid SHA-1`. + Frontends can use this file to validate imports after they + have been completed. + +Performance +----------- +The design of gfi allows it to import large projects in a minimum +amount of memory usage and processing time. Assuming the frontend +is able to keep up with gfi and feed it a constant stream of data, +import times for projects holding 10+ years of history and containing +100,000+ individual commits are generally completed in just 1-2 +hours on quite modest (~$2,000 USD) hardware. + +Most bottlenecks appear to be in foreign source data access (the +source just cannot extract revisions fast enough) or disk IO (gfi +writes as fast as the disk will take the data). Imports will run +faster if the source data is stored on a different drive than the +destination Git repository (due to less IO contention). + + +Development Cost +---------------- +A typical frontend for gfi tends to weigh in at approximately 200 +lines of Perl/Python/Ruby code. Most developers have been able to +create working importers in just a couple of hours, even though it +is their first exposure to gfi, and sometimes even to Git. This is +an ideal situation, given that most conversion tools are throw-away +(use once, and never look back). + + +Parallel Operation +------------------ +Like `git-push` or `git-fetch`, imports handled by gfi are safe to +run alongside parallel `git repack -a -d` or `git gc` invocations, +or any other Git operation (including `git prune`, as loose objects +are never used by gfi). + +gfi does not lock the branch or tag refs it is actively importing. +After the import, during its ref update phase, gfi tests each +existing branch ref to verify the update will be a fast-forward +update (the commit stored in the ref is contained in the new +history of the commit to be written). If the update is not a +fast-forward update, gfi will skip updating that ref and instead +prints a warning message. gfi will always attempt to update all +branch refs, and does not stop on the first failure. + +Branch updates can be forced with `--force`, but its recommended that +this only be used on an otherwise quiet repository. Using `--force` +is not necessary for an initial import into an empty repository. + + +Technical Discussion +-------------------- +gfi tracks a set of branches in memory. Any branch can be created +or modified at any point during the import process by sending a +`commit` command on the input stream. This design allows a frontend +program to process an unlimited number of branches simultaneously, +generating commits in the order they are available from the source +data. It also simplifies the frontend programs considerably. + +gfi does not use or alter the current working directory, or any +file within it. (It does however update the current Git repository, +as referenced by `GIT_DIR`.) Therefore an import frontend may use +the working directory for its own purposes, such as extracting file +revisions from the foreign source. This ignorance of the working +directory also allows gfi to run very quickly, as it does not +need to perform any costly file update operations when switching +between branches. + +Input Format +------------ +With the exception of raw file data (which Git does not interpret) +the gfi input format is text (ASCII) based. This text based +format simplifies development and debugging of frontend programs, +especially when a higher level language such as Perl, Python or +Ruby is being used. + +gfi is very strict about its input. Where we say SP below we mean +*exactly* one space. Likewise LF means one (and only one) linefeed. +Supplying additional whitespace characters will cause unexpected +results, such as branch names or file names with leading or trailing +spaces in their name, or early termination of gfi when it encounters +unexpected input. + +Date Formats +~~~~~~~~~~~~ +The following date formats are supported. A frontend should select +the format it will use for this import by passing the format name +in the `--date-format=<fmt>` command line option. + +`raw`:: + This is the Git native format and is `<time> SP <tz>`. + It is also gfi's default format, if `--date-format` was + not specified. ++ +The time of the event is specified by `<time>` as the number of +seconds since the UNIX epoch (midnight, Jan 1, 1970, UTC) and is +written as an ASCII decimal integer. ++ +The timezone is specified by `<tz>` as a positive or negative offset +from UTC. For example EST (which is typically 5 hours behind GMT) +would be expressed in `<tz>` by ``-0500'' while GMT is ``+0000''. ++ +If the timezone is not available in the source material, use +``+0000'', or the most common local timezone. For example many +organizations have a CVS repository which has only ever been accessed +by users who are located in the same location and timezone. In this +case the user's timezone can be easily assumed. ++ +Unlike the `rfc2822` format, this format is very strict. Any +variation in formatting will cause gfi to reject the value. + +`rfc2822`:: + This is the standard email format as described by RFC 2822. ++ +An example value is ``Tue Feb 6 11:22:18 2007 -0500''. The Git +parser is accurate, but a little on the lenient side. Its the +same parser used by gitlink:git-am[1] when applying patches +received from email. ++ +Some malformed strings may be accepted as valid dates. In some of +these cases Git will still be able to obtain the correct date from +the malformed string. There are also some types of malformed +strings which Git will parse wrong, and yet consider valid. +Seriously malformed strings will be rejected. ++ +If the source material is formatted in RFC 2822 style dates, +the frontend should let gfi handle the parsing and conversion +(rather than attempting to do it itself) as the Git parser has +been well tested in the wild. ++ +Frontends should prefer the `raw` format if the source material +is already in UNIX-epoch format, or is easily convertible to +that format, as there is no ambiguity in parsing. + +`now`:: + Always use the current time and timezone. The literal + `now` must always be supplied for `<when>`. ++ +This is a toy format. The current time and timezone of this system +is always copied into the identity string at the time it is being +created by gfi. There is no way to specify a different time or +timezone. ++ +This particular format is supplied as its short to implement and +may be useful to a process that wants to create a new commit +right now, without needing to use a working directory or +gitlink:git-update-index[1]. ++ +If separate `author` and `committer` commands are used in a `commit` +the timestamps may not match, as the system clock will be polled +twice (once for each command). The only way to ensure that both +author and committer identity information has the same timestamp +is to omit `author` (thus copying from `committer`) or to use a +date format other than `now`. + +Commands +~~~~~~~~ +gfi accepts several commands to update the current repository +and control the current import process. More detailed discussion +(with examples) of each command follows later. + +`commit`:: + Creates a new branch or updates an existing branch by + creating a new commit and updating the branch to point at + the newly created commit. + +`tag`:: + Creates an annotated tag object from an existing commit or + branch. Lightweight tags are not supported by this command, + as they are not recommended for recording meaningful points + in time. + +`reset`:: + Reset an existing branch (or a new branch) to a specific + revision. This command must be used to change a branch to + a specific revision without making a commit on it. + +`blob`:: + Convert raw file data into a blob, for future use in a + `commit` command. This command is optional and is not + needed to perform an import. + +`checkpoint`:: + Forces gfi to close the current packfile, generate its + unique SHA-1 checksum and index, and start a new packfile. + This command is optional and is not needed to perform + an import. + +`commit` +~~~~~~~~ +Create or update a branch with a new commit, recording one logical +change to the project. + +.... + 'commit' SP <ref> LF + mark? + ('author' SP <name> SP LT <email> GT SP <when> LF)? + 'committer' SP <name> SP LT <email> GT SP <when> LF + data + ('from' SP <committish> LF)? + ('merge' SP <committish> LF)? + (filemodify | filedelete)* + LF +.... + +where `<ref>` is the name of the branch to make the commit on. +Typically branch names are prefixed with `refs/heads/` in +Git, so importing the CVS branch symbol `RELENG-1_0` would use +`refs/heads/RELENG-1_0` for the value of `<ref>`. The value of +`<ref>` must be a valid refname in Git. As `LF` is not valid in +a Git refname, no quoting or escaping syntax is supported here. + +A `mark` command may optionally appear, requesting gfi to save a +reference to the newly created commit for future use by the frontend +(see below for format). It is very common for frontends to mark +every commit they create, thereby allowing future branch creation +from any imported commit. + +The `data` command following `committer` must supply the commit +message (see below for `data` command syntax). To import an empty +commit message use a 0 length data. Commit messages are free-form +and are not interpreted by Git. Currently they must be encoded in +UTF-8, as gfi does not permit other encodings to be specified. + +Zero or more `filemodify` and `filedelete` commands may be +included to update the contents of the branch prior to the commit. +These commands can be supplied in any order, gfi is not sensitive +to pathname or operation ordering. + +`author` +^^^^^^^^ +An `author` command may optionally appear, if the author information +might differ from the committer information. If `author` is omitted +then gfi will automatically use the committer's information for +the author portion of the commit. See below for a description of +the fields in `author`, as they are identical to `committer`. + +`committer` +^^^^^^^^^^^ +The `committer` command indicates who made this commit, and when +they made it. + +Here `<name>` is the person's display name (for example +``Com M Itter'') and `<email>` is the person's email address +(``cm@example.com''). `LT` and `GT` are the literal less-than (\x3c) +and greater-than (\x3e) symbols. These are required to delimit +the email address from the other fields in the line. Note that +`<name>` is free-form and may contain any sequence of bytes, except +`LT` and `LF`. It is typically UTF-8 encoded. + +The time of the change is specified by `<when>` using the date format +that was selected by the `--date-format=<fmt>` command line option. +See ``Date Formats'' above for the set of supported formats, and +their syntax. + +`from` +^^^^^^ +Only valid for the first commit made on this branch by this +gfi process. The `from` command is used to specify the commit +to initialize this branch from. This revision will be the first +ancestor of the new commit. + +Omitting the `from` command in the first commit of a new branch will +cause gfi to create that commit with no ancestor. This tends to be +desired only for the initial commit of a project. Omitting the +`from` command on existing branches is required, as the current +commit on that branch is automatically assumed to be the first +ancestor of the new commit. + +As `LF` is not valid in a Git refname or SHA-1 expression, no +quoting or escaping syntax is supported within `<committish>`. + +Here `<committish>` is any of the following: + +* The name of an existing branch already in gfi's internal branch + table. If gfi doesn't know the name, its treated as a SHA-1 + expression. + +* A mark reference, `:<idnum>`, where `<idnum>` is the mark number. ++ +The reason gfi uses `:` to denote a mark reference is this character +is not legal in a Git branch name. The leading `:` makes it easy +to distingush between the mark 42 (`:42`) and the branch 42 (`42` +or `refs/heads/42`), or an abbreviated SHA-1 which happened to +consist only of base-10 digits. ++ +Marks must be declared (via `mark`) before they can be used. + +* A complete 40 byte or abbreviated commit SHA-1 in hex. + +* Any valid Git SHA-1 expression that resolves to a commit. See + ``SPECIFYING REVISIONS'' in gitlink:git-rev-parse[1] for details. + +The special case of restarting an incremental import from the +current branch value should be written as: +---- + from refs/heads/branch^0 +---- +The `^0` suffix is necessary as gfi does not permit a branch to +start from itself, and the branch is created in memory before the +`from` command is even read from the input. Adding `^0` will force +gfi to resolve the commit through Git's revision parsing library, +rather than its internal branch table, thereby loading in the +existing value of the branch. + +`merge` +^^^^^^^ +Includes one additional ancestor commit, and makes the current +commit a merge commit. An unlimited number of `merge` commands per +commit are permitted by gfi, thereby establishing an n-way merge. +However Git's other tools never create commits with more than 15 +additional ancestors (forming a 16-way merge). For this reason +it is suggested that frontends do not use more than 15 `merge` +commands per commit. + +Here `<committish>` is any of the commit specification expressions +also accepted by `from` (see above). + +`filemodify` +^^^^^^^^^^^^ +Included in a `commit` command to add a new file or change the +content of an existing file. This command has two different means +of specifying the content of the file. + +External data format:: + The data content for the file was already supplied by a prior + `blob` command. The frontend just needs to connect it. ++ +.... + 'M' SP <mode> SP <dataref> SP <path> LF +.... ++ +Here `<dataref>` can be either a mark reference (`:<idnum>`) +set by a prior `blob` command, or a full 40-byte SHA-1 of an +existing Git blob object. + +Inline data format:: + The data content for the file has not been supplied yet. + The frontend wants to supply it as part of this modify + command. ++ +.... + 'M' SP <mode> SP 'inline' SP <path> LF + data +.... ++ +See below for a detailed description of the `data` command. + +In both formats `<mode>` is the type of file entry, specified +in octal. Git only supports the following modes: + +* `100644` or `644`: A normal (not-executable) file. The majority + of files in most projects use this mode. If in doubt, this is + what you want. +* `100755` or `755`: A normal, but executable, file. +* `120000`: A symlink, the content of the file will be the link target. + +In both formats `<path>` is the complete path of the file to be added +(if not already existing) or modified (if already existing). + +A `<path>` string must use UNIX-style directory seperators (forward +slash `/`), may contain any byte other than `LF`, and must not +start with double quote (`"`). + +If an `LF` or double quote must be encoded into `<path>` shell-style +quoting should be used, e.g. `"path/with\n and \" in it"`. + +The value of `<path>` must be in canoncial form. That is it must not: + +* contain an empty directory component (e.g. `foo//bar` is invalid), +* end with a directory seperator (e.g. `foo/` is invalid), +* start with a directory seperator (e.g. `/foo` is invalid), +* contain the special component `.` or `..` (e.g. `foo/./bar` and + `foo/../bar` are invalid). + +It is recommended that `<path>` always be encoded using UTF-8. + +`filedelete` +^^^^^^^^^^^^ +Included in a `commit` command to remove a file from the branch. +If the file removal makes its directory empty, the directory will +be automatically removed too. This cascades up the tree until the +first non-empty directory or the root is reached. + +.... + 'D' SP <path> LF +.... + +here `<path>` is the complete path of the file to be removed. +See `filemodify` above for a detailed description of `<path>`. + +`mark` +~~~~~~ +Arranges for gfi to save a reference to the current object, allowing +the frontend to recall this object at a future point in time, without +knowing its SHA-1. Here the current object is the object creation +command the `mark` command appears within. This can be `commit`, +`tag`, and `blob`, but `commit` is the most common usage. + +.... + 'mark' SP ':' <idnum> LF +.... + +where `<idnum>` is the number assigned by the frontend to this mark. +The value of `<idnum>` is expressed as an ASCII decimal integer. +The value 0 is reserved and cannot be used as +a mark. Only values greater than or equal to 1 may be used as marks. + +New marks are created automatically. Existing marks can be moved +to another object simply by reusing the same `<idnum>` in another +`mark` command. + +`tag` +~~~~~ +Creates an annotated tag referring to a specific commit. To create +lightweight (non-annotated) tags see the `reset` command below. + +.... + 'tag' SP <name> LF + 'from' SP <committish> LF + 'tagger' SP <name> SP LT <email> GT SP <when> LF + data + LF +.... + +where `<name>` is the name of the tag to create. + +Tag names are automatically prefixed with `refs/tags/` when stored +in Git, so importing the CVS branch symbol `RELENG-1_0-FINAL` would +use just `RELENG-1_0-FINAL` for `<name>`, and gfi will write the +corresponding ref as `refs/tags/RELENG-1_0-FINAL`. + +The value of `<name>` must be a valid refname in Git and therefore +may contain forward slashes. As `LF` is not valid in a Git refname, +no quoting or escaping syntax is supported here. + +The `from` command is the same as in the `commit` command; see +above for details. + +The `tagger` command uses the same format as `committer` within +`commit`; again see above for details. + +The `data` command following `tagger` must supply the annotated tag +message (see below for `data` command syntax). To import an empty +tag message use a 0 length data. Tag messages are free-form and are +not interpreted by Git. Currently they must be encoded in UTF-8, +as gfi does not permit other encodings to be specified. + +Signing annotated tags during import from within gfi is not +supported. Trying to include your own PGP/GPG signature is not +recommended, as the frontend does not (easily) have access to the +complete set of bytes which normally goes into such a signature. +If signing is required, create lightweight tags from within gfi with +`reset`, then create the annotated versions of those tags offline +with the standard gitlink:git-tag[1] process. + +`reset` +~~~~~~~ +Creates (or recreates) the named branch, optionally starting from +a specific revision. The reset command allows a frontend to issue +a new `from` command for an existing branch, or to create a new +branch from an existing commit without creating a new commit. + +.... + 'reset' SP <ref> LF + ('from' SP <committish> LF)? + LF +.... + +For a detailed description of `<ref>` and `<committish>` see above +under `commit` and `from`. + +The `reset` command can also be used to create lightweight +(non-annotated) tags. For example: + +==== + reset refs/tags/938 + from :938 +==== + +would create the lightweight tag `refs/tags/938` referring to +whatever commit mark `:938` references. + +`blob` +~~~~~~ +Requests writing one file revision to the packfile. The revision +is not connected to any commit; this connection must be formed in +a subsequent `commit` command by referencing the blob through an +assigned mark. + +.... + 'blob' LF + mark? + data +.... + +The mark command is optional here as some frontends have chosen +to generate the Git SHA-1 for the blob on their own, and feed that +directly to `commit`. This is typically more work than its worth +however, as marks are inexpensive to store and easy to use. + +`data` +~~~~~~ +Supplies raw data (for use as blob/file content, commit messages, or +annotated tag messages) to gfi. Data can be supplied using an exact +byte count or delimited with a terminating line. Real frontends +intended for production-quality conversions should always use the +exact byte count format, as it is more robust and performs better. +The delimited format is intended primarily for testing gfi. + +Exact byte count format:: + The frontend must specify the number of bytes of data. ++ +.... + 'data' SP <count> LF + <raw> LF +.... ++ +where `<count>` is the exact number of bytes appearing within +`<raw>`. The value of `<count>` is expressed as an ASCII decimal +integer. The `LF` on either side of `<raw>` is not +included in `<count>` and will not be included in the imported data. + +Delimited format:: + A delimiter string is used to mark the end of the data. + gfi will compute the length by searching for the delimiter. + This format is primarly useful for testing and is not + recommended for real data. ++ +.... + 'data' SP '<<' <delim> LF + <raw> LF + <delim> LF +.... ++ +where `<delim>` is the chosen delimiter string. The string `<delim>` +must not appear on a line by itself within `<raw>`, as otherwise +gfi will think the data ends earlier than it really does. The `LF` +immediately trailing `<raw>` is part of `<raw>`. This is one of +the limitations of the delimited format, it is impossible to supply +a data chunk which does not have an LF as its last byte. + +`checkpoint` +~~~~~~~~~~~~ +Forces gfi to close the current packfile and start a new one. +As this requires a significant amount of CPU time and disk IO +(to compute the overall pack SHA-1 checksum and generate the +corresponding index file) it can easily take several minutes for +a single `checkpoint` command to complete. + +.... + 'checkpoint' LF + LF +.... + +Packfile Optimization +--------------------- +When packing a blob gfi always attempts to deltify against the last +blob written. Unless specifically arranged for by the frontend, +this will probably not be a prior version of the same file, so the +generated delta will not be the smallest possible. The resulting +packfile will be compressed, but will not be optimal. + +Frontends which have efficient access to all revisions of a +single file (for example reading an RCS/CVS ,v file) can choose +to supply all revisions of that file as a sequence of consecutive +`blob` commands. This allows gfi to deltify the different file +revisions against each other, saving space in the final packfile. +Marks can be used to later identify individual file revisions during +a sequence of `commit` commands. + +The packfile(s) created by gfi do not encourage good disk access +patterns. This is caused by gfi writing the data in the order +it is received on standard input, while Git typically organizes +data within packfiles to make the most recent (current tip) data +appear before historical data. Git also clusters commits together, +speeding up revision traversal through better cache locality. + +For this reason it is strongly recommended that users repack the +repository with `git repack -a -d` after gfi completes, allowing +Git to reorganize the packfiles for faster data access. If blob +deltas are suboptimal (see above) then also adding the `-f` option +to force recomputation of all deltas can significantly reduce the +final packfile size (30-50% smaller can be quite typical). + +Memory Utilization +------------------ +There are a number of factors which affect how much memory gfi +requires to perform an import. Like critical sections of core +Git, gfi uses its own memory allocators to ammortize any overheads +associated with malloc. In practice gfi tends to ammoritize any +malloc overheads to 0, due to its use of large block allocations. + +per object +~~~~~~~~~~ +gfi maintains an in-memory structure for every object written in +this execution. On a 32 bit system the structure is 32 bytes, +on a 64 bit system the structure is 40 bytes (due to the larger +pointer sizes). Objects in the table are not deallocated until +gfi terminates. Importing 2 million objects on a 32 bit system +will require approximately 64 MiB of memory. + +The object table is actually a hashtable keyed on the object name +(the unique SHA-1). This storage configuration allows gfi to reuse +an existing or already written object and avoid writing duplicates +to the output packfile. Duplicate blobs are surprisingly common +in an import, typically due to branch merges in the source. + +per mark +~~~~~~~~ +Marks are stored in a sparse array, using 1 pointer (4 bytes or 8 +bytes, depending on pointer size) per mark. Although the array +is sparse, frontends are still strongly encouraged to use marks +between 1 and n, where n is the total number of marks required for +this import. + +per branch +~~~~~~~~~~ +Branches are classified as active and inactive. The memory usage +of the two classes is significantly different. + +Inactive branches are stored in a structure which uses 96 or 120 +bytes (32 bit or 64 bit systems, respectively), plus the length of +the branch name (typically under 200 bytes), per branch. gfi will +easily handle as many as 10,000 inactive branches in under 2 MiB +of memory. + +Active branches have the same overhead as inactive branches, but +also contain copies of every tree that has been recently modified on +that branch. If subtree `include` has not been modified since the +branch became active, its contents will not be loaded into memory, +but if subtree `src` has been modified by a commit since the branch +became active, then its contents will be loaded in memory. + +As active branches store metadata about the files contained on that +branch, their in-memory storage size can grow to a considerable size +(see below). + +gfi automatically moves active branches to inactive status based on +a simple least-recently-used algorithm. The LRU chain is updated on +each `commit` command. The maximum number of active branches can be +increased or decreased on the command line with `--active-branches=`. + +per active tree +~~~~~~~~~~~~~~~ +Trees (aka directories) use just 12 bytes of memory on top of the +memory required for their entries (see ``per active file'' below). +The cost of a tree is virtually 0, as its overhead ammortizes out +over the individual file entries. + +per active file entry +~~~~~~~~~~~~~~~~~~~~~ +Files (and pointers to subtrees) within active trees require 52 or 64 +bytes (32/64 bit platforms) per entry. To conserve space, file and +tree names are pooled in a common string table, allowing the filename +``Makefile'' to use just 16 bytes (after including the string header +overhead) no matter how many times it occurs within the project. + +The active branch LRU, when coupled with the filename string pool +and lazy loading of subtrees, allows gfi to efficiently import +projects with 2,000+ branches and 45,114+ files in a very limited +memory footprint (less than 2.7 MiB per active branch). + + +Author +------ +Written by Shawn O. Pearce <spearce@spearce.org>. + +Documentation +-------------- +Documentation by Shawn O. Pearce <spearce@spearce.org>. + +GIT +--- +Part of the gitlink:git[7] suite + diff --git a/git-resolve.html b/git-resolve.html index 2d2daac..9ea8e6e 100644 --- a/git-resolve.html +++ b/git-resolve.html
@@ -276,7 +276,7 @@ </div> <h2>DESCRIPTION</h2> <div class="sectionbody"> -<p>DEPRECATED. Use <tt>git-merge</tt> instead.</p> +<p>DEPRECATED and will be removed in 1.5.1. Use <tt>git-merge</tt> instead.</p> <p>Given two commits and a merge message, merge the <merged> commit into <current> commit, with the commit log message <message>.</p> <p>When <current> is a descendant of <merged>, or <current> is an @@ -299,7 +299,7 @@ </div> <div id="footer"> <div id="footer-text"> -Last updated 17-Jan-2007 23:27:40 UTC +Last updated 07-Feb-2007 05:52:26 UTC </div> </div> </body>
diff --git a/git-resolve.txt b/git-resolve.txt index 0925973..7fde665 100644 --- a/git-resolve.txt +++ b/git-resolve.txt
@@ -12,7 +12,7 @@ DESCRIPTION ----------- -DEPRECATED. Use `git-merge` instead. +DEPRECATED and will be removed in 1.5.1. Use `git-merge` instead. Given two commits and a merge message, merge the <merged> commit into <current> commit, with the commit log message <message>.
diff --git a/git.html b/git.html index 065f21a..1291cb8 100644 --- a/git.html +++ b/git.html
@@ -2288,7 +2288,7 @@ </div> <div id="footer"> <div id="footer-text"> -Last updated 06-Feb-2007 00:09:25 UTC +Last updated 07-Feb-2007 05:52:27 UTC </div> </div> </body>
